A Default First Order Family Weight Determination

Weighted Probability Distribution Voting (WPDV) is a newly designed machine learning algorithm, for which research is currently aimed at the determination of good weighting schemes. This paper describes a simple yet eeective weight determination procedure, which leads to models that can produce competitive results for a number of NLP classiication tasks. 1 The WPDV algorithm Weighted Probability Distribution Voting (WPDV) is a supervised learning approach to classiication. A case which is to be classiied is represented as a feature-value pair set: An estimation of the probabilities of the various classes for the case in question is then based on the classes observed with similar feature-value pair sets in the training data. To be exact, the probability of class C for F case is estimated as a weighted sum over all possible subsets F sub of F case : ^ P (C) = N (C) X F sub Fcase W F sub f req(C j F sub) f req(F sub) with the frequencies (freq) measured on the training data, and N (C) a normalizing factor such that P ^ P (C) = 1. In principle, the weight factors W F sub can be assigned per individual subset. For the time being, however, they are assigned for groups of subsets. First of all, it is possible to restrict the subsets that are taken into account in the model, using the size of the subset (e.g. F sub contains at most 4 elements) and/or its frequency (e.g. F sub occurs at least twice in the training material). Subsets which do not fulll the chosen criteria are not used. For the subsets that are used, weight factors are not assigned per individual subset either, but rather per \family", where a family consists of those subsets which contain the same combination of feature types (i.e. the same f i). The two components of a WPDV model, distributions and weights, are determined separately. In this paper, I will use the term training set for the data on which the distributions are based and tuning set for the data on the basis of which the weights are selected. Whether these two sets should be disjunct or can coincide is one of the subjects under investigation. 2 Family weights The various family weighting schemes can be classiied according to the type of use they make of the tuning data. Here, I use a very rough classiication, into weighting …